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The Maintenance Climate Assessment Survey (MCAS) was developed to 
proactively assess factors that contribute to a high reliability organization and strong 
safety climate. The 3 rd Marine Air Wing (MAW), which was seeking to proactively 
improve its safety posture requested the assistance of the School of Aviation Safety at the 
Naval Postgraduate School to examine its safety climate. Previous studies of the MCAS 
instrument have focused on the items and their relationship to the HRO based model of 
safety effectiveness components: process auditing, reward system, quality assurance, risk 
management, command and control, and communication/functional relationships. The 
present effort is the first attempt to consider the relationship between item component 
responses and demographic item responses. It evaluates 893 maintainer responses to the 
MCAS from 3 rd MAW and looks for measurable effects due to demographics. This study 
finds that the regression models constructed using the demographics as explanatory 
variables have very little utility in predicting scores for the components. This result 
allows planners the relief of using the demographics as a low priority issue. 
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EXECUTIVE SUMMARY 



Naval Aviation is a hazardous undertaking, but in spite of its inherent risk, its 
Class A Flight Mishap (FM) rate has been cut in half for each decade from 1950 to 1990. 
Over the last decade, however, the proportion of aircraft losses in which human error has 
been cited as a contributor has remained relatively constant. To address human factors 
issues in flight mishaps, the Human Factors Quality Management Board (HFQMB) was 
established in 1996. By using Mishap Data Analysis (MDA), Organizational 
Benchmarking (OB), and Command Safety Assessment (CSA), the efforts of the 
HFQMB resulted in a significant reduction in FM incidence from the perspective of 
aircrew operations. 

Although human error in maintenance is a smaller contributor, it has been shown 
to be a factor in nearly one in five Class A FMs. To address human error in maintenance, 
the Human Factors Analysis and Classification System - Maintenance Extension 
(HFACS-ME) was developed to classify error types in maintenance. Since Naval 
Aviation is shown to be a high reliability organization (HRO) defined as an organization 
that operates in hazardous environment with less than its fair share of accidents, it shares 
common characteristics with other HROs. These common characteristics are outlined in 
the Model of Organizational Safety Effectiveness (MOSE) and are process auditing, 
quality, reward system, risk management and command and control. Military aviation has 
communication/functional relationships as a sixth component. The augmented MOSE is 
the basis of the Maintenance Climate Assessment Survey (MCAS), which is used to 
evaluate the organizational safety climate from the perspective of the maintenance 
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personnel. The MCAS consists of six demographic items and 43 perception items. Each 
of the 43 perception items maps into a single component of the augmented MOSE. These 
questions are collapsed into six component scores for each respondent. 

Although MCAS has been shown to be an effective tool for evaluating the safety 
environment in a maintenance organization, demographic factors and their potential 
relationship with maintainer responses have not been investigated. This study evaluates 
MCAS responses from 894 maintenance personnel of the 3 rd MAW, and looks at how the 
demographic factors of maintenance personnel might be biasing the component scores of 
the MCAS. The results of this thesis are intended to further refine MCAS demographic 
factors and provide Squadron Commanders with insight into the construct of their 
maintenance personnel. 

The component scores are fitted using the demographics as explanatory factors. 
Univariate analysis is performed for each component using simple models without 
interaction and also with models using two-factor interactions. These models are then 
simplified in order to reduce the number of terms to a more manageable level. 

The results of this thesis show that up to two-factor interaction, the demographic 
factors of the MCAS poorly account for the variance in the responses. The reliance on 
subjective perception in the scoring is the cause of the large amount of variance. Since 
variance cannot be explained by the demographic factors, the MCAS appears to be 
demographically unbiased. Input from subject matter experts is used to refine the 
demographic factors. These revised factors are more usable for Squadron Commanders in 
that they provide more insight into the make up of the maintenance organization. 
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I. INTRODUCTION 



A. OVERVIEW 

Naval Aviation is a hazardous undertaking, but in spite of its inherent risk, its 
Class A Flight Mishap (FM) rate has been cut in half for each decade from 1950 and 
1990 (Naval Safety Center, 1997). Class A Mishaps are defined as Naval aircraft 
incidents resulting in death, permanent disability, or property loss or damage in excess of 
one million dollars (OPNAV 3750. 6Q, 1989). A flight mishap (FM) is defined as those 
mishaps in which there is $10,000 or greater DoD aircraft damage or loss of a DoD 
aircraft, and intent for flight for DoD aircraft existed at the time of the mishap. Other 
property damage, injury, or death may or may not have occurred. Naval Aviation 
consistently maintains high levels of operability coupled with less than its fair share of 
accidents (Goodrum, 1999). Naval Aviation also possesses the requisite characteristics of 
a high reliability organization (HRO): process auditing, reward system, quality, risk 
management, and command and control. For these reasons, Roberts (1988) labeled Naval 
Aviation an HRO. 

Although Naval Aviation is successful in reducing its Class A FM rate, over the 
last decade the proportion of aircraft losses in which human error has been cited as a 
contributor has remained relatively constant at four of five FMs (Naval Safety Center, 
2000). In 1996, a Human Factors Quality Management Board (HFQMB) is established 
after 17 Class A FMs occurred in only 75 days, climaxing when a Navy F-14 crashes into 
a Nashville, TN neighborhood, to address human factors issues related to mishaps 
(Nutwell & Sherman, 1997). The goal of the HFQMB is to cut the current Class A FM 
rate due to human error in half by year 2000 (HFQMB Charter, 1996). The HFQMB 



adopts three approaches to identify and target factors contributing to human error: 1) 
Mishap Data Analysis (MDA), 2) Organizational Benchmarking (OB), and 3) Command 
Safety Assessment (CSA). 

MDA establishes the development of HFACS, which is used to identify and 
prioritize human factors contributors to FMs. Among others, it determines inadequate 
supervision and aircrew violations are significant contributors (Shappel & Wiegman, 
1997). Using OB which explores programs which influence aircrew performance, the 
HFQMB determines use of feedback mechanisms in commercial airlines improve crew 
resource management training benefits (Nutwell & Sherman, 1997). Finally, a CSA 
survey, based on a model of HROs, is developed to determine a command’s safety 
posture from an aircrew perspective. This survey finds that 55% of the Navy respondents 
and 65% of the Marine Corps respondents feel that their commands are committed 
beyond what available resources can provide (Ciavarelli & Figlock, 1997). These 
combined efforts make significant progress toward the HFQMB ’s goal as evidenced by 
fiscal year 1999 being the safest year in Naval Aviation history in terms of Class A FM 
rate. 

Maintenance is shown to be a contributing factor in nearly one in five Class A 
FMs (Naval Safety Center, 2000). Additionally, during FY90-97, Class C FMs account 
for 75% of all maintenance related mishaps (MRMs). Maintenance is one area where 
hazards can be controlled and risk can be managed while an aircraft is on the ground. 
Much work is done in the field of human factors in maintenance safety for commercial 
airlines (“Human Factors in,” 2000). In 1988, the Aviation Safety Research Act (ASRA) 
mandates close study of aging aircraft structures and human factors affecting safety 
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(“History,” 2000). In the spirit of this mandated study, Boeing finds that incomplete 
installation (34%), damaged on installation (15%), improper installation (11%) and 
equipment not installed/missing (11%) were the top contributors in maintenance error 
(Komamiski, 2000). This investigation and classification of types of human error in 
maintenance leads to the development of Boeing’s Maintenance Error Decision Aid 
(MEDA), a system that aids operators and maintainers in the investigation and mitigation 
of maintenance related errors (Allen, Rankin, & Sargent, 1998). 

The ASRA is one of the precursors for the FAA’s current goal to reduce the fatal 
accident rate 80% by 2007 as compared to 1994-1996 baseline data (FAA, 1998). Several 
key initiatives are the stepping stones for this ultimate goal: 1) the development of a 
maintenance resource management system; 2) establishment of new training 
requirements; 3) implementation of technical advances in aircraft maintenance at repair 
stations; 4) enforcement of safety recommendations from the National Transportation 
Safety Board (NTSB); and 5) recommendations for aging systems maintenance. Little 
work until recently is done for military aviation, and recent efforts involve studying 
forms of maintenance error (Schmorrow, 1998) and the perceived maintenance safety 
climate (Baker, 1998). From a proactive perspective, efforts must be made to continue 
developing assessment tools to identify potential areas for risk management and control 
of conditions before a mishap occurs. 

Using the Maintenance Climate Assessment Survey (MCAS), Goodrum (1999) 
and Oneto (1999) are able to show the prototype survey effectively evaluates a 
maintainer’s perception of safety in maintenance operations. They also further refine the 
MCAS into a present 43 question format. One aspect of the MCAS yet to be addressed is 
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the demographic categories to determine their potential relationship with maintainer 
responses. Given the structure of maintenance organizations within aircraft communities 
vary, it is unlikely that any pair of samples will have equal proportions of all 
demographic variables. It is anticipated that individual demographic characteristics may 
influence MCAS responses and are therefore potentially biasing the results. By 
understanding the effects of the demographics, one can understand if they impact their 
organization’s safety climate. 

B. BACKGROUND 

The 3 rd Marine Air Wing (MAW) is a combat-ready expeditionary aviation force 
capable of short-notice worldwide employment to Marine Air Ground Task Force 
(MAGTF) fleet and unified commanders. It is composed of 28 squadrons divided into 
four Marine Air Groups (MAGs) based in Southern California and Arizona. Each MAG 
has its own combat mission: MAG 1 1 provides air support to MAGTF commanders; 
MAG- 13 provides close-air support, conducts armed reconnaissance, and assumes 
limited air-defense roles; MAG- 16 transports and resupplies Marine air and ground units; 
and MAG-39 provides utility helicopter support, close-in fire support, fire support 
coordination, aerial reconnaissance, observation and forward air control in aerial and 
ground escort operations during ship-to-shore movement and subsequent operations 
ashore. The aircraft used in these missions are AH-lWs, UH-lNs, CH-53s, CH-46Es, 
F/A-18DS, F/A-18S, AV-8s and C-130Ts. 

From 1990 to 1996, maintenance is a causal factor in 17 percent of all Naval 
Aviation class A FMs (Naval Safety Center, 1997). From April 1997 to July 1999, 
maintenance, maintenance personnel or maintenance depot is cited as a causal factor in 
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14 FMs (eight class C FMs, four class B FMs, and two class A FMs) experienced by 3 rd 
MAW. The Commander of 3 rd MAW requests the assistance of the School of Aviation 
Safety (SAS) at the Naval Postgraduate School, which in turn provides safety and risk 
management training to personnel, mishap data analysis, and administration of safety 
climate surveys to help locate problems in the organization. 

C. PROBLEM STATEMENT 

Human error in aviation is an issue that needs to be addressed, and it is recognized 
that the organization has an impact on factors that lead to it. Organizations that possess 
the attributes of a HRO tend to generate environments conducive to the reduction or 
control of human error and consequently experience fewer mishaps. Organizations 
aspiring towards the reduction of mishaps need to assess their safety posture as it relates 
to the attributes of HROs. The School of Aviation Safety at the Naval Postgraduate 
School has developed surveys to assess HRO characteristics in the operational 
environment for aircrew and maintenance personnel. 

The 3 rd MAW in an attempt to improve its safety posture enlists to have the 
School of Aviation Safety employ the MCAS survey to assess maintainer perception of 
HRO characteristics in its recent operations. These results are revealing, however in order 
to provide for better interpretation of the results, an exploration of the demographic 
variables is in order. This will help commanders to target more effectively specific areas 
of the organization that require attention. 

The current version of the MCAS is administered to the 3 rd MAW during the last 
half of 1999. Using statistical methods, the collected data is analyzed to assess 
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differences in responses that are correlated to differences in demographics. This thesis 
explores the following questions: 

1. Are there measurable demographic effects to the responses on the MCAS? 

2. Is there enough information in the demographics to be used in an adjustment 
process of the overall scores? 

3. Can the MCAS be refined further to either collapse or expand demographic 
factors? 

D. SCOPE AND LIMITATIONS 

Active duty U.S. Marine Corps Squadrons maintenance personnel of the 3 rd 
MAW are surveyed during the fall of 1 999. Only those squadrons with a representative 
number of respondents are used in the survey. Chapter II provides a basis for 
understanding human error, organizational safety culture, high reliability organizations 
and the assessment of a safety climate. Chapter III presents a discussion of the 
methodology used in this study. Results of data analysis are presented in Chapter IV. 
Chapter V summarizes previous chapters and provides conclusions and recommendations 
as they relate to the material. 

E. DEFINITIONS 

This thesis uses the following definitions (DON, 1989): 

Naval Aircraft . Refers to U.S. Navy, Naval Reserve, U.S. Marine Corps, and U.S. 
Marine Corps Reserve aircraft. 

Mishap . A Naval Aviation mishap is an unforeseen or unplanned event that directly 
involves naval aircraft, which result in $10,000 or greater cumulative damage to naval 
aircraft or personnel. The mishap is further divided into three classes based on the 



6 



amount of damage to the aircraft, property and personnel injury. The following are 
the definitions of the three classes: 

a. Class A . A mishap in which the total cost of property damage (including all 
aircraft damage) is $1,000,000 or greater; or a naval aircraft is destroyed or 
missing; or any fatality or permanent total disability of a person occurs with direct 
involvement of naval aircraft. 

b. Class B . A mishap in which the total cost of property damage (including all 
aircraft damage) is $200,000 or more but less than $1,000,000 and/or a permanent 
partial disability, and or the hospitalization of five or more personnel. 

c. Class C . A mishap in which the total cost of property damage (including all 
aircraft damage) is $10,000 or more but less than $200,000 and/or injury results in 
one or more lost workdays. 

Mishap rate . The total number of Class A,B and C mishaps per 100,000 flight hours. 
MCA$ . A 43-question survey used to gain insight into the maintenance community’s 
perception concerning aviation mishaps within the Navy and Marine Corps. 
HFAC$-ME . A taxonomic system used to classify causal factors that contribute to 
maintenance related mishaps. 

HRO. High-Reliability Organization, is an organization that operates in a hazardous 
environment, yet produces very low rate of accidents and incidents, operating effectively 
and safely and having the characteristics of leadership, sound management policies, 
procedure standardization, adequacy of resources and staffing, a defined system for risk 
management, and other factors. 



7 



THIS PAGE INTENTIONALLY LEFT BLANK 



8 



II. LITERATURE REVIEW 



A. HUMAN ERROR 

Reason (1990) defines error as a planned sequence that fails to achieve its 
intended outcome in the absence of external influence. He (1997) later describes error 
types as active or latent. Where the effects of active errors are often immediate and 
confined, latent conditions lie dormant until set off by a chain of local events and can be 
contributing factors in a variety of failures. This model of latent conditions and active 
failures is adopted by the Naval Safety Center to investigate Class A FMs with respect to 
aircrew error, and was the basis for the development of Human Factors Analysis and 
Classification System (HFACS) (Shappel & Wiegman, 1997). 



Activity 


| ‘Hands On’ 


Criticality 


Frequency 


Normal control 


Low 


Moderate 


High 


Emergency control 


Moderate 


High 


Low 


Maintenance-related 


High 


High 


High 



Table 1. Likelihood of Performance Problems in Given Activities. 



Reason (1997) also models human error in the scope of universal human activities 
(see Table 1) and the likelihood of performance problems within each of these types of 
activities. With HFACS, the Naval Safety Center is able to address human error in 
normal control and emergency control conditions. But Reason asserts that maintenance is 
the area with the highest likelihood of human error because maintenance related activities 
are almost exclusively reliant on human performance in the three areas of hands on, 
criticality and frequency. Even with progress in technology, human fallibility remains 
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constant (Reason, 1997) and with the frequency of planned maintenance compounded 
with the many pairs of fallible human hands working on exceptionally complicated 
systems, Reason’s model is a chilling prediction of 3 rd MAW’s situation. 

To address the maintenance related activity as a contributor to the total of human 
errors in FMs, Schmidt, Schmorrow and Hardee (1998) extend HFACS to specifically 
address the maintenance component of accident causation with the HFACS-Maintenance 
Extension (ME). The HFACS-ME expands upon Reason’s model of latent states and 
psychological precursors to unsafe acts. Reason (1990; 1997) differentiates these latent 
states in that they create the potential for human error. HFACS-ME classifies latent states 
in the maintenance environment, with three levels of error causation under four 
categories of conditions. The first order conditions are broad precursor categories 
(supervisory conditions, maintainer conditions, working conditions and maintainer acts) 
that are further divided into more specific precursors to human error of the second and 
third order (see Table 2). 

The causes of human error are many. Wickens, Gordon and Lui (1997) state that 
human error can be induced by “inattentiveness, poor work habits, lack of training, poor 
decision making, personality traits, social pressures, and so forth” (p. 427-428). Redmill 
& Rajan (1997) note that a common component in accidents is a worker’s loss of 
concentration which can be caused by “boredom, disinterest, distraction, or attempt to do 
two or more things at once” (p.12). Wickens, Gordon and Lui (1997) assert that the most 
common type of maintenance related error is that of omission. Considering that the nature 
of planned maintenance is to frequently disassemble, inspect then reassemble 
components, it is clear to see the high probability of human error in maintenance (Reason 
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1997). But as Perrow and Reason point out, the errors themselves are seldom isolated, but 
often a single event in a causal chain (Perrow, 1984), or encouraged, or at least not 
prohibited from occurring by latent conditions in the organization (Reason, 1997). 



First Order 


Second Order 


Third Order 


Supervisory Conditions 


Unforeseen 


Hazardous Operations 
Inadequate Documentation 
Inadequate Design 




Squadron 


Inadequate Supervision 
Inappropriate Operations 
Failed to Correct Problem 
Supervisory Violation 


Maintainer Conditions 


Medical 


Mental State 
Physical State 
Physical/Mental Limitation 




Crew Coordination 


Communication 

Assertiveness 

Adaptability/Flexibility 




Readiness 


Preparation/Training 

Qualification/Certification 

Violation 


Working Conditions 


Environment 


Lighting/Light 
Exposure/W eather 
Environmental Hazards 




Equipment 


Damaged 

Unavailable 

Dated/Uncertified 




Workspace 


Confining 

Obstructed 

Inaccessible 


Maintainer Acts 


Error 


Attention 

Memory 

Rule/Knowledge 

Skill 




Violation 


Routine 

Infraction 

Exceptional 



Table 2. HFACS-ME Levels of Error Causation. 
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B. ORGANIZATIONAL SAFETY CULTURE 



1. Definition 

Organizational culture is defined as shared values and beliefs that interact with an 
organization’s structures and control systems to produce behavioral norms (Uttal, 1983). 
All organizations have their own engineered culture whether good or bad. A safety 
culture is ideal for complex organizations and is defined as the product of individual and 
group values, attitude, competencies, and patterns of behavior that determine the 
commitment to, and the style and proficiency of, an organization’s health and safety 
programmes (Booth, 1993). 

2. Composition 

Redmill and Rajan (1997) state that there are three general aspects of safety 
culture: awareness, commitment and competence. Awareness must be present in all 
aspects of design, management and decision making. When a safety mishap does occur, 
it is commitment that drives the leadership of the organization to investigate and locate 
contributing factors and take immediate action to prevent another occurrence. 
Competence is a combination of education, training, professionalism and personality 
traits that are appropriate for a given task or job (Redmill & Rajan, 1997). 

Reason (1997) prefers the term informed culture, and he divides informed culture 
into four subcultures: reporting culture, just culture, flexible culture and learning culture. 
Reporting culture is “an organizational climate in which people are prepared to report 
their errors and near-misses.” Just culture is an “atmosphere of trust in which people are 
encouraged, even awarded, for provided essential safety-related information - but in 
which they are also clear about where the line must be drawn between acceptable and 
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unacceptable behavior.” Flexible culture involves “shifting from the conventional 
hierarchical mode to a flatter professional structure, where control passes to task experts 
on the spot, and then reverts back to the traditional bureaucratic mode once the 
emergency has passed. Such adaptability is an essential feature of the crisis-prepared 
organization.” Learning culture is “the willingness and the competence to draw the right 
conclusions from its safety information system, and the will to implement major reforms 
when their need is indicated.” 

Safety culture has powerful effects. First, it is self perpetuating where workers 
learn from each other and encourage each other to work in a manner consistent with the 
organization’s safety culture. People are quick to follow the example of coworkers, even 
if this means a lack of vigilance in safety (Redmill & Rajan, 1997). Wogalter, Allison, & 
McKenna (1989) assert that “people are extremely susceptible to social norms; they are 
likely to engage in safe or unsafe behaviors to the extent that others around them do so.” 
Safety culture is also self preserving as new workers learn to identify acceptable methods 
of accomplishing work and are able to pass those standards along to new employees, 
good or bad (Redmill & Rajan, 1997). 

C. HIGH RELIABILITY ORGANIZATIONS 

1. Definition 

Roberts (1990) and Libuser (1994) explain that High-Reliability Organizations 
(HROs), organizations that operate in a hazardous environment, yet produce very low 
rates of accidents and incidents, which operate effectively and safely have certain key 
characteristics in common. Reason (1997) calls them organizations with less than their 
fair share of accidents, and “highly complex, technology-intensive organizations that 
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must operate, as far as humanly possible, to a failure-free standard.” He also explains 
that HROs manage complex technologies that have very low tolerance for error, yet 
maintain the flexibility to successfully function in environments of extreme intensity. 

2. Characteristics of HROs 

Examples of HROs are the nuclear power industry, petrochemical industry, and 
airline industry. Additionally, Figlock (1998) identifies Naval aviation is an HRO. 
Although diverse in purpose, Roberts and Libuser believe these organizations share 
several common characteristics: leadership style, management policies, procedures 
standardization, superior training, a reward system that recognizes safety achievement, 
adequacy of resources and staffing, effective management of risks associated with 
hazardous operations, and other factors. 

HROs have a requisite variety. As Weick (1987) states, having diverse people 
from diverse backgrounds and experiences builds requisite variety that is required for 
relatively simple humans to operate complex systems. Additionally, this diversity is 
essential in problem solving, as individuals will approach the same problem uniquely, so 
that the collective contribution is greater than any one individual’s input. 

HROs typically exhibit a high degree of training. Weick (1987) notes that 
“training for the operation of high reliability systems is often tough and demanding so 
that the faint of heart and the incompetent are weeded out.” This is because HROs are not 
afforded the luxury of trial and error. Training is often in the form of simulation and 
stories. Stories have a big affect on the reliability of an HRO by lending experience to the 
inexperienced: 
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The basic idea is that a system which values stories, story tellers, and 
storytelling will be more reliable than a system that derogates these 
substitutes for trial and error. A system that values stories and storytelling 
is potentially more reliable because people know more about their system, 
know more of the potential errors that might occur, and they are more 
confident that they can handle those errors that do occur because they 
know that other people have already handled similar errors (Weick, 1987, 
p. H3). 

By sharing the experiences of skilled personnel, novices do not have to learn from their 

own mistakes and are also granted the insight of the skilled. 

HROs appear bureaucratic and uneventful on the surface. There is a strict chain of 

command in place that dictates policy, procedure and environment. This strong 

centralization is apparent during periods of relatively low intensity, but as intensity of 

operating increases, the true nature of the HRO is revealed, where flexibility, delegation, 

improvisation and technical expertise dominate (Reason, 1997; Weick, 1987). This is 

how HROs can be simultaneously centralized and decentralized. Responsibility and 

judgment remain centralized while creativity, improvisation and unsupervised problem 

solving become decentralized in environments of high intensity (Weick, 1987). 

Reliability in HROs is another deceptive aspect. Weick (1987) calls reliability a 

“dynamic non-event,” meaning a great deal of effort goes into ensuring nothing happens. 

This dynamicism is based on the belief that reliability is fleeting and systems tend to 

move to states of unreliability unless constantly maintained: 

Part of the mindset for reliability requires chronic suspicion that small 
deviations may enlarge, a sensitivity that may be encouraged by a more 
dynamic view of reliability (Weick, 1987, p. 1 19). 

It is the organizational culture for personnel to look for problems before they happen. 

Weick continues that because of the invisibility of the dynamics behind reliability, there 
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is a perception that reliability is easily achievable and is only noticed in the presence of a 
breakdown. 

D. ASSESSING SAFETY CLIMATE 

1. Safety Space 

Reason (1997) states that organizations can be mapped into a safety space which 
is a continuum of degrees of susceptibility to accidents (see Figure 1). Organizations with 
higher resistance will generally have fewer mishaps while organizations with higher 
vulnerability will generally experience more mishaps. No organization is immune. 
Chance, unforeseen circumstances, failures in defenses and human error can cause even 
the most resistant organizations to experience accidents. Within the safety space, currents 
tend to push organizations away from the extremes of resistance or vulnerability and 
toward the center, a compromise between the two. If an organization has the desire to 
become more resistant, it must swim upstream. 




Figure 1. Organizational Safety Space. 
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The effort required to move in the direction of increased resistance must be put 
into reactive and proactive measures. Reason (1997) contends that investigating mishaps 
to find causal factors to be addressed is not even half the battle. To effectively move the 
organization, mishap investigation must be coupled with the identification of conditions 
needing correction, and regular checks. 

2. MOSE and MCAS 

Libuser’s (1994) current Model of Organizational Safety Effectiveness (MOSE) is 
based on work by Roberts and is a categorization of the common characteristics of 
HROs. These characteristics are mapped into five components: 1) Process Auditing (PA) 
- checks by members to identify hazards; 2) Reward System (RS)- expected rewards or 
disciplinary action used to shape behavior; 3) Quality Assurance (QA)- promotion of 
quality performance; 4) Risk Management (RM)- system to identify hazards and control 
operational risks; and 5) Command and Control (CC)- safety climate, leadership 
effectiveness /policies, and procedures for mitigating risks. These components are very 
similar to the aspects of Reason’s (1997) informed culture (Table 3). 



Libuser’s MOSE Components 


Reason ’s Informed Culture 


Process Auditing (PA) 


Leaning Culture 


Reward System (RS) 


Just Culture 


Quality Control (QA) 


Reporting Culture 


Risk Management (RM) 


Flexible Culture 


Command and Control (CC) 


Flexible Culture 



Table 3. Comparison of Libuser’s MOSE and Reason’s Informed Culture. 
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Ciavarelli and Figlock (1997) adapt the MOSE for use in Naval Aviation using 
practices and terminology of that environment and develop the Command Safety 
Assessment, a survey that addresses each of the MOSE categories from the viewpoint of 
the aircrewman. This survey is administered to 1,254 aviators revealing that 
organizational and supervisory issues are seen by aircrewmen as impacting flight safety. 

The Maintenance Climate Assessment Survey (MCAS) is the product of the 
implementation of the MOSE and CSA in a maintenance context. Baker (1998) starts by 
reducing 155 candidate questions to 67 items that specifically addressed aviation 
maintenance. Augmenting Libuser’s (1994) five category MOSE model with a sixth 
category, Communication/Functional Relationships, Baker (1998) modifies the CSA to 
look at aviation safety from the point of view of the maintenance person. Using 
regression techniques. Baker is able to further reduce the survey into a compact 35 item 
form, with almost all questions mapped to a single category of the augmented MOSE. 

Goodrum (1999) and Oneto (1999) show that the MCAS is a valid tool to 
accurately assess an aviation maintenance environment, but note that some items in the 
survey need restructuring. Oneto notes that these items address more than one category of 
the MOSE. Their inputs help change the MCAS to its current 43 item format (see 
Appendix A). While Goodrum and Oneto are able to show content validity in the MCAS, 
there is much left to examine. Since there is no known or accepted measure for MCAS 
results and providing feedback to the concerned squadrons, an effort is underway to 
explore concurrent validity in the survey on a per question basis (Schmidt, personal 
communication). Questions that have a low response mean are noted as areas that need 
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attention, corresponding to a particular category of the MOSE, and which part of the 
HRO needs closer examination. 

Additionally, by looking at squadron mean scores to the survey and available 
mishap data, Harris (personal communication) is looking at the MCAS predictive validity 
in the incidence of mishaps within a squadron based on adjusted mean scores. While the 
work by Harris seeks to broaden the scope and applicability of the MCAS, this study 
explores internal aspects of the survey. By separating responses by demographics, this 
thesis will further explore the attitude of the Naval aviation maintenance person with 
respect to safety. 

3. Instrument Design and Demographics 

The design of the MCAS is a cross-sectional one time look at a maintenance 
organization (see Appendix A). It is a self-administered questionnaire that polls 
respondent perception about the safety of their working environment at all levels. 
Demographic items preserve anonymity by excluding personal questions which could be 
linked to individuals allowing personnel the freedom to express their true opinion to the 
perception items (Oneto, 1999). 

Though there are many methods for data collection from people, surveys, when 
designed properly, are very effective for recording respondent scores based on a 
particular model (Fink & Kosecoff, 1985). MCAS responses are forced using a five point 
Likert scale (see Appendix A). Using a forced scale instead of subjective comments 
allows for rapid compilation of data and analyzation of responses as numerical values, 
which is convenient for numerical scoring and comparisons of scores between 
respondents, or groups of respondents (Fink & Kosecoff, 1985). The grouping is 
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facilitated by the six demographic items of the survey (Baker, 1998). By being able to 
group types of maintainers independent of their scores, it is possible to investigate for 
unequal perceptions across demographic groups. 

The demographic items in the MCAS capture aspects of the maintainer within the 
organization such as experience, and rank, and ignore personal information such as age, 
race, sex and ethnicity (Baker, 1998). Although it would be difficult to capture every 
possible combination of demographic factors, the MCAS is able to capture most job types 
from most aircraft communities. Each of the demographic line items in the MCAS 
represents a simple factor that might influence the scoring. It is unknown which of these 
demographics constitutes a valid or invalid factor. 

E. SUMMARY 

HROs are complex and dynamic by nature but are not impervious to unsafe trends 
in human performance. Perrow (1984) points out that regardless of an organization’s 
structure and nature, “normal accidents” will continue to occur. With HROs, these 
accidents tend to occur less frequently, but the consequences of the accidents tend to be 
large in magnitude. These organizations must put effort into swimming upstream through 
Reason’s safety space towards increased resistance to accidents. This happens through 
reactive measures like mishap analysis and proactive measures to identify “pathogenic 
conditions” (Reason, 1997). Though reactive measures are in place, proactive measures 
are coming up to speed. 

Two critical parallel developments in organization safety theory in regards to 
aviation safety are the development of effective taxonomies (e.g., HFACS-ME) and the 
identification and accurate modeling of HROs (e.g., augmented MOSE). The 
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development of CSA and MCAS are steps to link the taxonomies and the organization 
models. Since MOSE parallels Reason’s informed culture, it is possible with MCAS to 
identify those conditions that are not conducive to safety and take proactive measures to 
move the organization towards increased resistance to accidents in the safety space. Since 
3 rd MAW is a HRO, CSA and MCAS allow for the identification of the MOSE 
components that require attention. 

The MCAS has been revised into a more usable form, and has been validated, 
showing that the individual items do address specific MOSE components. This thesis is 
another step into revising the MCAS further by looking at the validity of the demographic 
factors. Mapping the demographic factors into the MOSE component scores will show if 
the factors are relevant or can be removed from the survey. Additionally, if these factors 
do not account for the variance in scores, the survey is not asking the correct 
demographic questions and will require further revision. 
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III. METHODOLOGY 



A. RESEARCH APPROACH 

The intent of this study is to assess the maintained s perception of safety in his or 
her work environment. This research involves the use of analysis techniques to partition 
the collected data into smaller groups based on demographics then investigate differences 
in responses among the groups. If statistical differences are found, a comparison between 
the group mean and a particular squadron mean shows how a particular demographic 
differs from the rest of the squadron. Conversely, it shows which demographics have 
response means that are more reflective of the squadron means. 

B. DATA COLLECTION 

1. Subjects 

Surveys are administered to 977 officers and enlisted personnel responsible for 
Naval Aviation maintenance. These subjects come from squadrons and maintenance units 
of the 3 rd Marine Air Wing located at MCAS Miramar, CA, Camp Pendleton, CA, and 
MCAS Yuma, AZ. The aircraft represented are the AH-1 “Super Cobra,” UH-1 “Huey,” 
CH-53 “Super Stallion,” CH-46E “Sea Knight,” F/A-18D “Night Attack Hornet,” F/A-18 
“Hornet,” AV-8B “Harrier,” and the COOT “Hercules.” 

Additionally, subject matter experts are interviewed about what they consider to 
be important demographic information about the personnel in their maintenance 
organizations. Subject matter experts are military aviators with at least eight years active 
duty service. The results of these interviews are given in the next section. 
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2. Instrument 



The MCAS is a self-administered, group survey consisting of two parts: 1) 
demographics: 2) perception. Part I captures demographic factors of each subject: 
community, squadron, rank, years of aviation maintenance experience, work center, and 
shift. There are eight choices available for community, with an additional option of 
“other.” The squadron factor records the three-digit squadron designator. Embedded in 
these two factors is aircraft type (seventh factor). Rank is divided into four levels of 
enlisted personnel and three levels of officer personnel. Years of aviation maintenance 
experience is partitioned into seven levels. Work center or shop, is divided into eight 
shops with the option of “other.” Shift divides subjects into dayshift or nightshift. 

Part II captures subject perception of his or her work environment. There are 42 
items, each of which is mapped into a single augmented MOSE component: process 
auditing (six questions), reward system (eight questions), quality assurance (six 
questions), risk management (nine questions), command and control (eight questions), 
and communication/functional relations (six questions). Each perception item asks 
subjects to rank a specific safety related activity or aspect of their organization using a 
five point Likert rating scale with verbal anchors as follows: Strongly Disagree. Disagree, 
Neutral, Agree, Strongly Agree. When completed, the items for each MOSE component 
are averaged to attain six composite scores, each one corresponding to the subject’s rating 
of that particular augmented MOSE component for his or her organization. 

3. Procedure 

The survey is administered on site and in a group setting at the various 
participating Squadrons of the 3 rd MAW. Additionally, the survey is given in 
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conjunction with a scheduled maintenance safety presentation on human factors issues in 
aviation. The Squadrons are in various stages of training and operational tasking at the 
time of the survey being administered. The variety of operational tasking with which the 
squadrons are simultaneously involved during the administration of the MCAS accounts 
for much of the variance in the number of surveys collected from each squadron. 
Potential MCAS respondents are briefed on maintenance issues, the survey and its 
purpose and questions that arise pertaining to the survey are answered by the survey 
administer. Respondents fill out the surveys using scannable computer forms. The 
surveys are then immediately collected upon completion to allow for maximum 
accountability. 

For personal interviews, subject matter experts report what they think are the 
important factors when considering the demographics of personnel in their maintenance 
organizations. Notes are taken during these interviews and a list is compiled from the 
responses of the personnel interviewed. This list represents potential MCAS demographic 
items. 

C. DATA ANALYSIS 

1. Data Tabulation 

Survey results were compiled into a database using a scanning machine, then 
imported into Microsoft Excel. The spreadsheet consists of rows of respondents and 
columns of survey items (both demographic and survey items). Demographic items 
record mainly bivariate and multivariate responses, such as squadron (aircraft type 
embedded), rank, and years experience. Survey item responses were assigned a numerical 
value of 1 through 5 corresponding to the Likert scale, with higher values being assigned 
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to more positive responses (strongly agree) and lower values assigned to more negative 
responses (strongly disagree). Each of the questions in the survey addresses one of the 
six MOSE components. Items addressing similar MOSE components were collapsed into 
an average score for that particular component. Any items that were missing values were 
excluded and not averaged into the component score. No weighting is assigned to items 
in the event of a missing item score. The three demographic response items 
corresponding to the respondent’s squadron number were collapsed into a single coded 
value. 

2. Statistical Analysis 

Microsoft Excel is used to provide summary statistics and initial familiarization 
with the data. The data were cleansed by removing subject responses with omitted 
demographic items leaving 894 responses. Items corresponding to augmented MOSE 
components were averaged for each respondent, leaving six scores along with 
demographic response items. If subjects omitted a perspective item response, the 
component score is averaged for the completed items of that component. No weighting 
for omitted perspective items is administered. Histograms of the demographic make up of 
the data are constructed. The data are then exported to ARC for its powerful graphing 
capabilities. Initial scatterplot matrices of each component over all demographic factors 
revealed no linear, exponential, or power trend, although scatterplots matrices of the 
components over themselves revealed a linear trend. 

The data are then transported to MathSoft S-plus for analysis. Categorical 
demographics are coded as factors and the aov function is implemented to fit the six 
component scores based on demographic factors. Six models (one for each component) 
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are constructed for a regression without interaction among factors. Six additional models 
are fit for a regression looking at two factor interaction. Each model is then simplified by 
using S-plus to remove unimportant terms. Models are compared and similar models for 
some of the components are found. Three term interaction models are not explored in this 
thesis. 
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IV. RESULTS 



A. SIMPLE MODELS WITHOUT INTERACTION 

A scatterplot matrix is constructed to see if there are any trends in the data. This is 
done without designating the independent variables as factors. The scatterplot shows that 
there is some relationship between the component scores, but little information about the 
demographics is revealed. Figure 3 is a scatterplot matrix for PA versus each of the 
demographic factors. Visual inspection reveals a possible relationship between PA and 
Rank and Total years of Aviation experience. 




Figure 1. Scatterplot of PA versus Demographic Factors. 

Three dimensional bar plots for the components versus each factor are constructed 
to see if there are any visual clues as to some type of relationship between the factors and 
the response. No linear, log linear or exponential relationship is visible. However, the 
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plots seem compatible with those of normally distributed histograms for each level of 
factor, all centered in the same approximate region of the component score (e.g. Figure 
4). 




Figure 2. Three Dimensional Bar Plot of PA Scores. PA scores 
are plotted against different levels of Total Years of Aviation 
Maintenance Experience. 

Since there is no indication that a transformation of the data is required due to 
visible trends, linear models for the six individual components are fit against 
untransformed factors without interaction using the aov function in S-plus. Model 
checking plots are constructed and case 219 is shown to have very high influence in all 
models. Case 219 is an E-6/7 with 15-20 years of aviation maintenance experience, 
works the day shift in “other” work center, in a VMH squadron. Although there is no 
significance test associated with Cook’s Distance, case 219 is deleted from the data set 
due to its unusually high influence and new models were fit (see Appendix C). 

Model checking plots are constructed to check the fit of the models with case 219 
removed. The scatter plot of the data with the regression superimposed reveals what 
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might be a slight upward trend in the response for all components (see Appendix D). The 
histograms of the residuals show that the distribution of the residuals appear to be normal 
(see Appendix E), and the scatterplot of the residuals versus the fitted values shows no 
discemable pattern in the residuals (see Appendix F). The QQ-plot shows that the 
residuals are thin at both tails for all models (see Appendix G), but the normal shape is 
tenable. 



Component 


R 2 


<7 


PA 


0.1095 


0.5529 


QA 


0.1219 


0.6327 


RS 


0.1427 


0.5896 


RM 


0.1869 


0.5690 


CC 


0.1280 


0.6349 


CR 


0.1233 


0.6846 



Table 4. R-squared and a for Models without Interaction. 



Values for the coefficient of determination, R 2 , show that these models account 
but poorly for the variance in the data. The best model is RM, accounting for less than 
19% of the total variance (see Table 4). The model with the lowest R~ is PA with only 
1 1% of the total variance explained. Values for standard error, <r, indicate that there is a 
large spread in the response values. For example, with a perfect R 2 of 1, the model for PA 
tells us that 68% of the respondents score between 3.31 and 4.42, and that 96% score 
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between 2.76 and 4.97. Since the only possible scores are between 1 and 5, relatively 
high values for a are not much help with understanding the data. 

The results from the analysis of variance from each model causes rejection of the 
null hypothesis that all of the coefficients in the model are zero, accepting the alternate 
hypothesis that at least one coefficient is not equal to zero (see Appendix H). 
Additionally, the models for PA, CC and CR fail the lack of fit test indicating that the 
shape of the fit is not correct. These models do not do well in describing the data. 

B. MODELS WITH TWO FACTOR INTERACTION 

Two-factor interaction models are constructed for all of the components using S- 
plus to see if more of the variance in the data can be modeled and model checking plots 
are constructed. The scatterplot of the data with the superimposed regression shows a 
linear trend in the response against the factors (see Appendix I), and the distribution of 
the residuals appears to be normal (see Appendix J). The scatterplot of the residuals 
versus the fitted values shows no discemable pattern in the residuals (see Appendix K). 
The QQ-plots show strange behavior at values close to zero, but that they are close to 
being normal for all components (see Appendix L). R 2 and d are given in Table 5. 

These two term interaction models are better at explaining more of the variance in 
the data as indicated by the values for the coefficients of determination, however there is 

very little reduction in the values for the standard error. This improvement in the values 

*) . 

for R' comes at the cost of increased complexity in the models. While the models without 
interaction have 43 terms, the two term interaction models have 343 terms. By adding 
300 terms to the model, 300 degrees of freedom are lost resulting in no significant 
improvement in the standard error. 
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Component 


~R t ~ 


<7 


PA 


0.4813 


0.5246 


QA 


0.4832 


0.6034 


RS 


0.5046 


0.5572 


RM 


0.5207 


0.5431 


CC 


0.4968 


0.5996 


CR 


0.4850 


0.6523 



Table 5. R 2 and a for Two Factor Interaction Models. 

Analysis of variance on the two factor interaction models causes rejection of the 
null hypothesis that all of the coefficients are equal to zero. The alternative hypothesis is 
accepted meaning that at least one coefficient is not equal to zero. Present in the anova 
tables is evidence that some of the terms are not necessary in the model and that 
simplification is possible. 

C. REDUCED MODELS WITHOUT INTERACTION 

The step function in S-plus is used to subtract terms from the simple models in an 
effort to simplify the models without losing too much of the information they provide. S- 
plus accomplishes this by using Akaike’s information criterion which is of the form: 

A1C = -2/ogL(* m +i, I * 1 , ...,x m ) + 2r 

where r is the total number of estimated parameters. The AIC is a value that penalizes a 
model for having high complexity when compared to simpler models with fewer terms. 
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The step function is applied to the simple models without interaction and the results are 
given in Appendix N. The reduced models have between 15 and 33 terms compared to 43 
terms for the original models. Model checking plots are constructed with no significant 
graphical differences between the reduced simple models and the simple models (see 
Appendices O-R). Values for the coefficient of determination and standard error are 
given in Table 6. As expected, less of the total variance is explained by the reduced 
models, and the value for a increases. 



Component 


R^ 


a 


PA 


0.0915 


0.5532 


QA 


0.1002 


0.6345 


RS 


0.1240 


0.5925 


RM 


0.1703 


0.5711 


CC 


0.1071 


0.6365 


CR 


0.0784 


0.6906 



Table 6. R 2 and <j for Reduced Models without Interaction. 

D. REDUCED TWO FACTOR INTERACTION MODELS 

The step function in S-plus is applied to the two factor interaction models and the 
results are given in Appendix S. The data were fit to the new models and model checking 
plots are constructed and given in Appendices T-W. The scatterplot of the data with the 
superimposed regression line indicates that there might be a linear relationship between 
the factors and the score, but most of the data looks like a point cloud. The histograms of 
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the residuals have the appearance of a normal distribution. The predicted values versus 
the residuals have no pattern and the QQ-plots look normal. 

The reduced models have significantly decreased values for R 2 with little change 
in values for <r. These models use between 33 and 59 terms which is a significant 
simplification over the 343 term models, but a lot of the explanation of the total variance 
is lost in the transition. Table 6 summarizes R' and cf for the reduced models. 



Component 


R 2 


a 


PA 


0.1098 


0.5496 


QA 


0.1228 


0.6287 


RS 


0.1854 


0.5802 


RM 


0.2453 


0.5534 


CC 


0.1747 


0.6225 


CR 


0.1342 


0.6779 



Table 7. R 2 and a for Reduced Two Factor Models with Interaction. 



E. COMPARING MODELS 

The S-plus anova function is applied to pairs of models to see if they are 
statistically different. The results are given in Appendix U. At a = 0.05, eight pairs of 
models were found to be statistically similar, with the highest similarity between simple 
and reduced two term interaction models for PA and QA (p- Value > 0.99). Most of the 
model pairs are statistically different. 
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Choosing the most appropriate model for a complex data set is compromise 
between a model that explains enough of the data while being simple enough to use. The 
two factor interaction models are too complicated to be practical with 343 terms. The rest 
of the models are not different enough to distinguish in practice, so the models with the 
fewest terms are the preferred models, which in this case are the reduced simple models. 
Realistically, none of the models are useful. Not enough of the total variance in the data 
is explained nor is the range of expected scores reduced to a useful level. 

F. INTERVIEWS 

Subject matter experts interviewed agree that while all of the factors in the survey 
are important, additional factors could be added to more effectively group personnel and 
provide more information about the maintenance organization. They also think that 
dividing total years of aviation maintenance experience into two items, years worked in 
MOS and years worked outside of MOS is necessary to clarify that Total years of 
aviation maintenance experience is not simply time on active duty. The experts also 
indicate factors that look at levels of education and training and levels of morale and 
motivation are important and should be included in the demographic items of the survey. 
Responses from subject matter experts are given in Appendix Y. 
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V. CONCLUSIONS 



A. FINDINGS 

The results of this thesis show that at the first level of interaction, the 
demographic factors of the MCAS poorly account for the variance in the responses. The 
models constructed using linear regression and analysis of variance do not capture the 
responses of the surveyed population, showing that the demographic factors have low 
utility in data analysis. While analysis of variance shows that the models are preferred to 
no model at all, in use the models are too complex and do not provide enough insight into 
the surveyed group. 

Since the component scores are subjective perceptions, there is no correct score to 
any of the perception items. The reliance on the human component in the scoring is the 
cause of the large amount of variance, and since variance cannot be explained by the 
demographic factors, the MCAS appears to be demographically unbiased. The three 
dimensional bar plots of the component scores versus the levels of factors seem to 
support this (see Figure 4). Either the MCAS has insignificant biasing across factors or 
the present factors do not correctly group respondents to allow the biasing to be 
conclusively measured. 

Although the demographics do not effectively group respondents, they do provide 
information about the demographic composition of the surveyed group. This information 
alone can be useful to commanders in understanding the substance of their squadrons. To 
make these items more useful, changes to the MCAS demographic items based on the 
responses from subject matter experts are recommended below. 
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B. RECOMMENDATIONS 



With the MCAS moving to the internet, it will be possible to use more 
demographic factors than what is currently constrained by the layout of scannable 
computer response sheets. The MCAS demographic items should be changed to include 
the following items: 

1) Check the box corresponding to your community: 

2) Type in your unit number. 

3) Type in the number of months have you been with your current squadron. 

4) Type in the total number of maintenance activities to which you have been 

assigned. 

5) Type in the number of deployments you have made. 

6) Check the box corresponding to your rank. 

7) Type in the number years have you worked in your MOS. 

8) Type in the number of years have you worked outside of your MOS. 

9) Type in any supervisory designations that you hold. 

10) Check the box corresponding to your work center. 

1 1) Check the box corresponding to your shift. 

12) Have you attended A School? 

With these new questions, further analysis can be conducted to investigate for 
valid factors that properly describe the data in the responses, in addition to investigating 
if personal performance makes a difference in scoring. Additionally, MCAS could be 
modified to gauge safety climates in other military activities such as military ordnance 
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handling facilities and flight deck operations by adjusting the demographic items to suit 
those specific activities. 
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APPENDIX A. 43-ITEM MAINTENANCE CLIMATE ASSESSMENT SURVEY 



MAINTENANCE CLIMATE ASSESSMENT SURVEY (MCAS) 

Purpose : The MCAS was designed to capture maintainer perceptions of maintenance operations as they 
relate to safety. Your responses help guide Naval Aviation’s on-going efforts to reduce aviation related 
mishaps. Thank you in advance for your participation ! 

Directions : Do not write on this form. Fill in all of your responses using the computer sheet provided. 

Fill in each box that corresponds to your response completely using a pencil. This is not a timed event, so 
answer each question carefully and honestly. Individual responses will not be reported, only compiled 
results will be provided to each squadron. 

Part I - Demographics has six items requesting unit and biographical data. This information will aid in the 
response analysis. NO attempts will be made to identify individuals. 

Part II - Perceptions has 43 questions pertaining to the maintenance operations. Please choose the response 
to each item that most correctly reflects your honest opinion. Responses are: 

A- Strongly Agree B- Agree C- Neutral D- Disagree E- Strongly Disagree 

Part I- Demographics 

Line 1 Fill in the numbered circle corresponding to your community? 

(l)VMGR (2) VMA (3) VMFA (4) HMT (5) HMM 

(6) VMAQ (7) HMH (8) VMH (9) Other 

Line 2-4 Fill in the circles corresponding to your squadron number 

Line 5 Fill in the numbered circle corresponding with you rank 

(1) El-3 (2) E4-5 (3) E6-7 (4) E8-9 (5)W01-4 (6)01-03 (7)04-5 

Line 6 Fill in the numbered circle corresponding to your total years 

of Aviation Maintenance experience 
(1) <1 (2) 1-2 (3)3-5 (4)6-10 (5) 11-15 (6) 15-20 (7)20+ 

Line 7 Fill in the numbered circle corresponding to your work center 

(1) Power Plants (2) Airframes (3) Survival (4) Quality Assurance 
(5) Ordnance (6) Avionics (7) MAINT Control (8) Line (9) Other 

Line 8 Fill in the numbered circle corresponding to your shift 

(1) Day (2) Night 



Part II Perceptions 

Fill in the lettered circle that corresponds with your response to each item. 









SA 


A 


N 


D 


SD 


1 . 


The command 


adequately reviews and updates safety. 


(A) 


(B) 


(C) 


(D) 


(E) 








SA 


A 


N 


D 


SD 


2. 


The command 


monitors maintainer qualifications and has 


(A) 


(B) 


(C) 


(D) 


(E) 




a program that 


targets training deficiencies. 


SA 


A 


N 


D 


SD 


3. 


The command 


uses safety and medical staff to identify/ 


(A) 


(B) 


(C) 


(D) 


(E) 
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manage personnel at risk. 

4. CDIs/QARs routinely monitor maintenance evolutions. 

5. Tool control is taken seriously in the command and 
support equipment licensing is closely monitored. 

6. Signing off personnel qualifications are taken seriously. 

7. Our command climate promotes safe maintenance. 



8. Supervisors discourage SOP, NAMP, or other procedural 
violations and encourage reporting safety concerns. 

9. Peer influence discourages SOP, NAMP, or other 
violations and individuals feel free to report them. 

10. Violations of SOP, NAMP, or other procedures are not 
common in this command. 

1 1. The command recognizes individual safety achievement 
through rewards and incentives. 

12. Personnel are comfortable approaching supervisors 
about personal problems/illness 

1 3. Safety NCO, QAR, and CDI, are sought after billets. 



14. Unprofessional behavior is not tolerated in the command 



15. The command has a reputation for quality maintenance 
and sets standards to maintain quality control. 

16. QA and Safety are well respected, and are seen as 
essential to mission accomplishment. 

1 7. QARs/CDIs sign-off after required actions are complete 
and are not pressured by supervisors to sign-off. 

1 8. Maintenance on detachments is of the same quality 
as that at home station. 

19. Required publications/tools/equipment are available, 
current/serviceable, and used. 

20. QARs are helpful, and QA is not “feared” in my unit. 



21. Multiple job assignments and collateral duties adversely 
affect maintenance. 



SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 



A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 



N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 



D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 



SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 
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22. Safety is part of maintenance planning, and additional 
training/support is provided as needed. 

23. Supervisors recognize unsafe conditions and manage 
hazards associated with maintenance and the flight-line. 

24. I am provided adequate resources, time, personnel to 
accomplish my job. 

25. Personnel turnover does not negatively impact the 
command’s ability to operate safely. 

26. Supervisors are more concerned with safe maintenance 
than the flight schedule, and do no permit cutting corners. 

27. Day/Night Check have equal workloads, and staffing is 
sufficient on each shift. 

28. Supervisors shield personnel from outside pressures and 
are aware of individual workload. 

29. Based upon my command's current assets/manning it 
is not over-committed. 

30. My command temporarily restricts maintainers who are 
having a problems. 

31. Safety decisions are made at the proper levels and 
work center supervisor decisions are respected. 

32. Supervisors communicate command safety goals and 
are actively engaged in the safety program. 

33. Supervisors set the example for following to 
maintenance standards and ensure compliance. 

34. In my command safety is a key part of all maintenance 
operations, and all are responsible/accountable for safety. 

35. Safety education and training are comprehensive and 
effective. 

36. All maintenance evolutions are properly briefed, 
supervised, and staffed by qualified personnel. 

37. Maintenance Control is effective in managing all 
maintenance activities. 

38. Good communication exists up/down the chain of 
command. 

39. I get all the information I need to do my job safely. 



SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 

SA 

(A) 



A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 

A 

(B) 



N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 

N 

(C) 



D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 

D 

(D) 



SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 

SD 

(E) 
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SA 


A 


N 


D 


SD 


40. 


Work center supervisors coordinate their actions. 


(A) 


(B) 


(C) 


(D) 


(E) 






SA 


A 


N 


D 


SD 


41. 


My command has effective pass-down between shifts. 


(A) 


(B) 


(C) 


(D) 


(E) 






SA 


A 


N 


D 


SD 


42. 


Maintenance Control troubleshoots/resolves gripes before 


(A) 


(B) 


(C) 


(D) 


(E) 




flight. 
















SA 


A 


N 


D 


SD 


43. 


Maintainers are briefed on potential hazards associated 


(A) 


(B) 


(C) 


(D) 


(E) 



with maintenance activities. 
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APPENDIX B. MODEL OF SAFETY EFFECTIVENESS COMPONENTS. 



COMPONENT 1: PROCESS AUDITING 

1. The command adequately reviews and updates safety practices. 

2. The command monitors maintainer qualifications and has a program that targets 
training deficiencies. 

3. The command uses safety and medical staff to identify/manage personnel at risk. 

4. CDIs/QARs routinely monitor maintenance evolutions. 

5. Tool Control is taken seriously in the command and support equipment licensing is 
closely monitored. 

6. Signing personal qualifications are taken seriously. 

COMPONENT 2: Reward System and Safety Climate 

1 . Our command climate promotes safe maintenance and flight operations. 

2. Supervisors discourage SOP, NAMP or other procedure violations and encourage 
reporting safety concern. 

3. Peer influence discourages SOP, NAMP or other violations and individuals feel free 
to report them. 

4. Violations of SOP, NAMP or other procedures are not common in this command. 

5. The command recognizes individual safety achievement through rewards and 
incentives. 

6. Personnel are comfortable approaching supervisors about personal problems/illness. 

7. Safety NCO, QAR, and CDI, are sought after billets. 

8. Unprofessional behavior is not tolerated in the command. 

COMPONENT 3: QUALITY ASSURANCE 

1. The command has a reputation for quality maintenance and has set standards to 
maintain quality control. 

2. QA and Safety are well respected, and are seen as essential to mission 
accomplishment. 

3. QARs/CDIs sign-off after required actions are complete and are not pressured by 
supervisors to sign-off. 

4. Maintenance on detachments is the same quality as that at home station. 

5. Required publications/tools/equipment are available, current/serviceable, and used. 

6. QARs are helpful, and QA is not “feared” in my unit. 
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COMPONENT 4: RISK MANAGEMENT 



1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 



Multiple job assignments and collateral duties adversely affect maintenance. 

Safety is part of maintenance planning, and additional training/support is provided as 
needed. 

Supervisors recognize unsafe conditions and manage hazards associated with 
maintenance and the flight line. 

I am provided adequate resources, time, personnel to accomplish my job. 

Personnel turnover does not negatively impact the command’s ability to operate 
safely. 

Supervisors are more concerned with safe maintenance than the flight schedule, and 
do not permit cutting comers. 

Day/Night Check have equal workloads, and staffing is sufficient on each shift. 
Supervisors shield personnel from outside pressures and are aware of individual 
workload. 

Based upon my command’s current assets/manning it is not over-committed. 



COMPONENT 5: COMMAND AND CONTROL 



1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 



My command temporarily restricts maintainers who are having problems. 

Safety decisions are made at the proper levels, work center supervisors decisions are 
respected. 

Supervisors communicate command safety goals and are actively engaged in the 
safety program. 

Supervisors set the example for following to maintenance standards and ensure 
compliance. 

In my command safety is a key part of all maintenance operations and all are 
responsible/accountable for safety. 

Safety education and training are comprehensive and effective. 

All maintenance evolutions are properly briefed, supervised, and staffed by qualified 
personnel. 

Maintenance control is effective in managing all maintenance activities. 



COMPONENT 6: COMMUNICATION / FUNCTIONAL RELATIONSHIPS 

1. Good communication exists up/down the chain of command. 

2. I get all the information I need to do my job safely. 

3. Work center supervisors coordinate their actions. 

4. My command has effective pass-down between shifts. 

5. Maintenance Control troubleshoots/resolves gripes before flight. 

6. Maintainers are briefed on potential hazards associated with maintenance activities. 
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Cook's Distance 



APPENDIX C. INFLUENCE PLOTS FOR AUGMENTED MOSE 
COMPONENTS 



PA 



RS 





Index 



QA 



RM 




0 200 400 600 800 

Index 




Index 



CC 



CR 




Index 
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APPENDIX D. SIMPLE MODELS: SCATTERPLOTS WITH SUPERIMPOSED 
REGRESSION LINE 



PA 



RS 





Fitted : (lactor(Commumty) ♦ fador(Squadron) + fador(Rank) + fador(Tot Years) «■ tactor(Shop) + lador(Sh Fined : (tador(Community) + tador(Squadron) + fador(Rank) » tactor(Tot Years) ♦ tador(Shop) + tador(S*> 



QA 



RM 





Fitted : (fador(Community) ♦ factor(Squadron) ♦ taetor(Rank) ♦ faclor(Tot. Years) + lador(Shop) + factor(Sh Fitted : (faclor(Commumty) ♦ lactor(Squadron) * tacior(Rank) «■ taclor{Tot Years] + lador(Shop) * fador($h 



CC 



CR 





Fitted : {faclor(Communily) ♦ tador(Squadron) ♦ fador(Rank) + «aclor(Toi Years) + lador(Shop) ♦ fador(Sh Fittod . (fador(Commumty) ♦ tador(Squadron) ♦ tador(Rank) ♦ fador(Tor Years) ♦ tador(Shop) ♦ tador(Sh 
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APPENDIX E. SIMPLE MODELS: HISTOGRAMS OF RESIDUALS 



PA 



QA 



o 

o 



o 



RS 



-2 




-1 0 1 
resid(pa aov) 




RM 




- 2-10 1 
resid(qa.aov) 



2 




resid(cc.aov) resid(craov) 



51 



THIS PAGE INTENTIONALLY LEFT BLANK 



52 



APPENDIX F. SIMPLE MODELS: RESIDUALS VERSUS FITTED 
VALUESWITH LOYVESS SMOOTHING 



PA QA 





RS 



RM 





rs.aovjfitled.valuos 



rm aov$fmed values 



CC 



CR 



o 




o o 
o ° 

—l 1 1 1 1 1 

3.0 3.2 3 4 3.6 3.8 4.0 4.2 

cc.aov$f rtl ed values 




O Ooo 



—| 1 1 : 1 1 r T- 

2.8 3.0 3.2 3 4 3.6 3.8 4.0 4.2 

cr aov$Utie<l.values 
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APPENDIX G. SIMPLE MODELS: QQ-PLOTS 



PA 



QA 





<*79 

<690 

, r ‘ '1 1 !— 

-3 - 2-10 1 

Quantiles ol Standard Normal 



RS 



RM 





<690 

! 1 1 1 

-3 - 2-10 1 

Quantiles of Standard Normal 



cc 



CR 










-1 0 



Quantiles of Standard Normal 



Quantiles of Standard Normal 
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APPENDIX H. SIMPLE MODELS: ANALYSIS OF VARIANCE 



PA 





Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


8.3925 


1.198929 


3.922101 


0.0003198 


factor (Squadron) 


13 


8.0763 


0.621257 


2.032343 


0.0160499 


factor (Rank) 


6 


6.0359 


1.005978 


3.290893 


0.0033082 


factor (Tot . Years ) 


6 


3.4403 


0.573388 


1.875746 


0.0821892 


factor (Shop) 


9 


3.2828 


0.364753 


1.193231 


0.2957039 


factor (Shift) 


1 


2.7153 


2.715335 


8.882780 


0.0029608 


Residuals 


850 


259.8325 


0.305685 






QA 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


8.3910 


1.198710 


2.994574 


0.0041284 


factor (Squadron) 


13 


14.3600 


1.104618 


2.759519 


0.0007443 


factor (Rank) 


6 


12 .2469 


2.041156 


5.099143 


0.0000372 


factor (Tot. Years) 


6 


7.6351 


1.272514 


3.178950 


0.0043234 


factor (Shop) 


9 


3.9649 


0.440543 


1.100548 


0.3596172 


factor (Shift) 


1 


0.6428 


0.642779 


1.605767 


0.2054349 


Residuals 


850 


340.2498 


0.400294 






RS 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


3.7699 


0.538562 


1.549360 


0.1471909 


factor (Squadron) 


13 


12.7604 


0.981569 


2.823824 


0.0005582 


factor (Rank) 


6 


17.8319 


2.971978 


8.549925 


0.0000000 


factor (Tot . Years) 


6 


7.6649 


1.277490 


3.675142 


0.0013048 


factor (Shop) 


9 


5.4436 


0.604846 


1.740050 


0.0761483 


factor (Shift) 


1 


1.7291 


1.729082 


4.974302 


0.0259869 


Residuals 


850 


295.4624 


0.347603 






RM 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


15.9912 


2.284458 


7.056521 


0.0000000 


factor (Squadron) 


13 


11.8830 


0.914080 


2.823525 


0.0005590 


factor (Rank) 


6 


19.2158 


3.202630 


9.892685 


0.0000000 


factor (Tot .Years) 


6 


10.5421 


1.757022 


5.427311 


0.0000161 


factor (Shop) 


9 


4.8880 


0.543110 


1.677627 


0.0901813 


factor (Shift) 


1 


0.7475 


0.747503 


2.308980 


0.1290003 


Residuals 


850 


275.1766 


0.323737 






cc 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


9.8503 


1.407189 


3.490669 


0.0010676 


factor (Squadron) 


13 


12.2826 


0.944816 


2.343707 


0.0045142 


factor (Rank) 


6 


15.1026 


2.517097 


6.243904 


0.0000020 


factor (Tot .Years) 


6 


6.7176 


1.119607 


2.777295 


0.0111256 


factor (Shop) 


9 


3.5477 


0.394188 


0.977822 


0.4567736 


factor (Shift) 


1 


2.8159 


2.815870 


6.985039 


0.0083707 


Residuals 


850 


342.6594 


0.403129 






CR 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


12.4423 


1.777478 


3.792880 


0.0004599 


factor (Squadron) 


13 


11.0908 


0.853140 


1.820477 


0.0361248 


factor (Rank) 


6 


18.6478 


3.107964 


6.631942 


0.0000007 


factor (Tot . Years) 


6 


9.4231 


1.570522 


3.351265 


0.0028616 


factor (Shop) 


9 


3.7036 


0.411516 


0.878115 


0.5443707 


factor (Shift) 


1 


0.7389 


0.738887 


1.576677 


0.2095850 


Residuals 


850 


398.3402 


0.468636 
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APPENDIX I. TWO FACTOR INTERACTION MODELS: SCATTERPLOTS 
WITH SUPERIMPOSED REGRESSION LINE 



PA 



QA 



aw o o odo 

O O CO D CCPODOCD o O 
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o ovc ood as o o o 
0 0000 o 

0 o o cd aamaxD o cro o 
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OOO OOODCHODOO&O 
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0 ODO IHWanD 0 oo 
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o OP IT TIT w qnoooopwo o 
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O 0 
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o 


QA 
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1 1 I 


<xxxt> c maxMWBOXBKiM <m> ooo 
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0 0 0 
o 
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2 3 4 5 1 2 3 4 S 

Fined : (laclor(Communrty) ♦ lactor(Squadron) lactor(Rank) ♦ tactor(Tot.Years) * factor(Shop) lactor(Shil Fined : (laclor(Commumty) ♦ factor(Squadron) * factor(Rank) ♦ factor(Tot. Years) * laclor(Shop) ♦ tactor(Shr 



RS 



RM 





Fined (factor(Commumty) + laclor(Squadron) ♦ lactor(Rank) + factor(Tot Years) ♦ factor(Shop) ♦ 1actor(Shii Fitted (facior(Commumty) ♦ lactor(Squadron) ♦ tactor{Rank) 1actor(Tot. Years) ♦ lactor(Shop) ♦ factor(Snn 



CC 



CR 





2 3 4 5 12345 

Fined : (lactor(Commumty) ♦ lactor(Squadron) «■ Jactor(Rank) ♦ 1actor(Tot Years) ♦ lactor(Shop) ♦ lactor{Shii Fitted (tactor(Commumty) ♦ factor(Squadron) ♦ lactor(Rank) * factor(Tot Years) * lactor(Shop) ♦ lactor{Shr 
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100 300 0 100 300 0 50 100 200 



APPENDIX J. TWO FACTOR INTERACTION MODELS: HISTOGRAMS OF 
RESIDUALS 




-3 -2 



;1 0 1 2 
resid(cc.2.aov) 



- 2-10 1 2 
resid(cr.2.aov) 
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APPENDIX K. TWO FACTOR INTERACTION MODELS: RESIDUALS 
VERSUS FITTED VALUES WITH LOESS SMOOTHING 



PA 



QA 





qa.2 aovSlitied values 



RS 



RM 




CC 



CR 





cc. 2. aov5! m ed. values 
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APPENDIX L. TWO FACTOR INTERACTION MODELS: QQ-PLOTS 



PA 



QA 




RS 



RM 





CC 



CR 
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APPENDIX M. TWO FACTOR INTERACTION MODELS: ANALYSIS OF 
VARIANCE 



PA 





Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


8.3925 


1 .198929 


4.356653 


0.0001027 


factor (Squadron) 


13 


8.0763 


0.621257 


2.257518 


0.0068172 


factor (Rank) 


6 


6.0359 


1.005978 


3 . 655510 


0.0014394 


factor (Tot . Years) 


6 


3.4403 


0.573388 


2.083571 


0.0534953 


factor ( Shop) 


9 


3.2828 


0.364753 


1.325436 


0.2203764 


factor (Shift) 


1 


2.7153 


2.715335 


9.866954 


0.0017732 


factor (Community) : factor (Rank) 


27 


8.5470 


0.316557 


1.150301 


0.2755360 


factor (Community) : factor (Tot .Years) 


32 


5.5049 


0.172029 


0.625117 


0.9480617 


factor (Community) : factor ( Shop) 


46 


16.8120 


0.365478 


1.328069 


0.0778513 


factor (Community) : factor (Shift) 


5 


4 . 8802 


0.976038 


3 . 546715 


0.0036412 


factor (Squadron) : factor (Rank) 


27 


7.1875 


0.266205 


0.967334 


0.5134968 


factor (Squadron) : factor (Tot . Years) 


37 


16.5409 


0.447050 


1.624486 


0.0126771 


factor (Squadron) : factor (Shop) 


56 


23 .4663 


0.419042 


1.522709 


0.0108588 


factor (Squadron) : factor (Shift) 


6 


3.7496 


0.624928 


2.270855 


0.0356434 


factor (Rank) : factor (Tot . Years) 


8 


3.4152 


0.426899 


1.551259 


0.1367135 


factor (Rank) : factor ( Shop) 


15 


5 . 1111 


0.340743 


1.238189 


0.2382294 


factor (Rank) : factor (Shift) 


2 


0.4713 


0.235625 


0.856213 


0.4253331 


factor (Tot . Years) : factor ( Shop) 


29 


8 . 8454 


0.305014 


1.108357 


0.3199687 


factor(Tot. Years) : factor ( Shift) 


3 


0.7165 


0.238846 


0.867915 


0.4574968 


factor (Shop) : factor ( Shift) 


7 


3 .2273 


0.461047 


1 . 675347 


0.1124277 


Residuals 


550 


151.3572 


0.275195 






QA 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


8.3910 


1 . 198710 


3.292186 


0.0019394 


factor (Squadron) 


13 


14.3600 


1 . 104618 


3.033770 


0.0002431 


factor (Rank) 


6 


12.2469 


2.041156 


5.605916 


0.0000116 


factor (Tot . Years) 


6 


7.6351 


1.272514 


3.494886 


0.0021165 


factor (Shop) 


9 


3.9649 


0.440543 


1.209925 


0.2860380 


factor (Shift) 


1 


0.6428 


0.642779 


1.765354 


0.1845090 


factor (Community) : factor (Rank) 


27 


10.0763 


0.373196 


1.024962 


0.4313503 


factor (Community) : factor (Tot .Years) 


32 


10.2932 


0.321663 


0.883428 


0.6538426 


factor (Community) : factor (Shop) 


46 


23.9700 


0.521087 


1.431136 


0.0362521 


factor (Community) : factor (Shift) 


5 


11.2388 


2.247751 


6.173318 


0.0000140 


factor (Squadron) : factor (Rank) 


27 


17.7225 


0. 656389 


1.802734 


0.0083195 


factor (Squadron) : factor (Tot . Years) 


37 


15.4891 


0.418624 


1.149727 


0.2540719 


factor (Squadron) : factor (Shop) 


56 


22.7429 


0.406123 


1.115393 


0.2703154 


factor (Squadron) : factor ( Shift ) 


6 


3.5604 


0.593400 


1.629737 


0.1365990 


factor (Rank) : factor (Tot . Years) 


8 


5.2739 


0. 659237 


1.810557 


0.0725066 


factor (Rank) : factor (Shop) 


15 


7.4882 


0.499213 


1.371060 


0.1561216 


factor (Rank) : factor (Shift) 


2 


1.7580 


0.879015 


2 . 414162 


0.0903895 


factor (Tot . Years) : factor ( Shop) 


29 


7.0553 


0.243286 


0.668170 


0.9075797 


factor (Tot .Years) : factor (Shift) 


3 


0.8364 


0.278789 


0.765679 


0.5136070 


factor (Shop) : factor (Shift) 


7 


2.4857 


0.355105 


0.975276 


0.4483208 


Residuals 


550 


200.2591 


0.364107 






RS 


Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


factor (Community) 


7 


3 .7699 


0.538562 


1.734800 


0.0984123 


factor ( Squadron) 


13 


12.7604 


0.981569 


3 .161802 


0.0001365 


factor (Rank) 


6 


17.8319 


2.971978 


9.573249 


0.0000000 


factor (Tot .Years) 


6 


7.6649 


1.277490 


4.115013 


0.0004719 


factor (Shop) 


9 


5.4436 


0.604846 


1.948314 


0.0432207 


factor ( Shift) 


1 


1.7291 


1.729082 


5.569667 


0.0186225 


factor (Community) : factor (Rank) 


27 


8.6010 


0.318554 


1.026117 


0.4297560 


factor (Community) : factor (Tot . Years) 


32 


5.9900 


0.187188 


0.602964 


0.9597519 


factor (Community) : factor (Shop) 


46 


19.4596 


0.423035 


1.362667 


0.0607806 


factor (Community) : factor ( Shift) 


5 


6.8711 


1.374220 


4.426596 


0.0005820 


factor (Squadron) : factor (Rank) 


27 


12.1684 


0.450680 


1.451719 


0.0672413 


factor (Squadron) : factor (Tot . Years) 


37 


14 . 7707 


0.399208 


1.285918 


0.1242051 


factor (Squadron) : factor ( Shop) 


56 


26.1193 


0.466416 


1.502405 


0.0132179 


factor (Squadron) : factor (Shift) 


6 


4.7507 


0.791781 


2.550461 


0.0191331 


factor(Rank) : factor (Tot . Years ) 


8 


3.5565 


0.444564 


1.432017 


0.1800429 


factor (Rank) : factor (Shop) 


15 


4 . 1334 


0.275561 


0.887628 


0.5783215 


factor (Rank) : factor ( Shift ) 


2 


1 . 6918 


0.845879 


2.724720 


0.0664497 


factor (Tot. Years) : factor (Shop) 


29 


10.6388 


0.366855 


1 . 181702 


0.2373524 


factor (Tot . Years ) : factor ( Shift ) 


3 


2.3787 


0.792884 


2.554013 


0.0546501 


factor (Shop) : factor ( Shift) 


7 


3.5872 


0.512457 


1 . 650712 


0.1187383 


Residuals 


550 


170.7454 


0.310446 
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RM 

factor (Community) 
factor ( Squadron) 
factor (Rank) 
factor (Tot . Years) 
factor (Shop) 
factor (Shift) 
factor (Community) : factor (Rank) 
factor (Community) : factor (Tot . Years) 
factor (Community) : factor ( Shop) 
factor (Community) : factor (Shift) 
factor ( Squadron) : factor ( Rank) 
factor (Squadron) : factor (Tot . Years) 
factor (Squadron) : factor ( shop) 
factor ( Squadron) : factor (Shift) 
factor (Rank) : factor (Tot . Years) 
factor (Rank) : factor ( Shop) 
factor (Rank) : factor ( Shift) 
factor (Tot . Years) : factor ( Shop) 
factor (Tot . Years) : factor (Shi ft) 
factor (Shop) : factor (Shift) 
Residuals 

CR 

factor ( Community) 
factor (Squadron) 
factor (Rank) 
factor (Tot . Years) 
factor ( Shop) 
factor ( Shift) 
factor (Community) : factor (Rank) 
factor (Community) : factor (Tot . Years) 
factor (Community) : factor ( Shop) 
factor (Community) : factor (Shift) 
factor ( Squadron) : factor (Rank) 
factor (Squadron) : factor (Tot . Years) 
factor ( Squadron) : factor ( Shop) 
factor (Squadron) : factor (Shift) 
factor(Rank) : factor (Tot . Years) 
factor (Rank) : factor ( Shop) 
factor (Rank) : factor (Shift) 
factor (Tot . Years) : factor ( Shop) 
factor (Tot .Years) : factor ( Shift) 
factor (Shop) : factor ( Shift ) 
Residuals 

cc 

factor (Community) 
factor ( Squadron) 
factor (Rank) 
factor (Tot . Years) 
factor (Shop) 
factor (Shift) 
factor (Community) : factor (Rank) 
factor (Community) : factor (Tot .Years) 
factor (Community) : factor (Shop) 
factor (Community) : factor (Shift) 
factor ( Squadron) : factor (Rank) 
factor (Squadron) : factor (Tot . Years) 
factor (Squadron) : factor (Shop) 
factor (Squadron) : factor (Shift) 
factor (Rank) : factor (Tot . Years) 
factor (Rank) : factor ( Shop) 
factor (Rank) : f actor ( Shi f t) 
factor (Tot . Years) : factor (Shop) 
factor (Tot . Years) : factor (Shi ft) 
factor(Shop) : factor (Shift) 
Residuals 



Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


15.9912 


2.284458 


7.74590 


0.0000000 


11.8830 


0.914080 


3.09937 


0.0001810 


19.2158 


3.202630 


10.85914 


0.0000000 


10.5421 


1.757022 


5. 95752 


0.0000048 


4.8880 


0.543110 


1.84152 


0.0583837 


0.7475 


0.747503 


2 . 53455 


0.1119530 


6.1172 


0.226562 


0.76820 


0.7946819 


7.0593 


0.220602 


0.74799 


0.8421223 


15.9480 


0 . 346696 


1.17554 


0.2054702 


12.6939 


2.538785 


8.60824 


0.0000001 


10.8087 


0.400322 


1.35737 


0.1093488 


12.4608 


0.336778 


1.14191 


0.2636165 


20.8355 


0.372062 


1.26155 


0.1036550 


5.6544 


0.942395 


3 . 19537 


0.0043116 


2.4103 


0.301291 


1.02158 


0.4183172 


4.4618 


0.297452 


1.00857 


0.4442584 


2.6308 


1 .315404 


4.46013 


0.0119820 


8.4000 


0.289655 


0.98213 


0.4942530 


1.2055 


0.401827 


1.36247 


0.2533738 


2.2818 


0.325976 


1.10528 


0.3581240 


162.2087 


0.294925 







Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


12 . 4423 


1.777478 


4 . 177394 


0.0001697 


11.0908 


0.853140 


2.005033 


0.0185347 


18.6478 


3.107964 


7 . 304274 


0.0000002 


9.4231 


1.570522 


3 . 691009 


0.0013214 


3.7036 


0.411516 


0.967136 


0.4662855 


0.7389 


0.738887 


1.736518 


0.1881293 


10.0529 


0.372329 


0.875041 


0.6494108 


10.0165 


0.313016 


0.735645 


0.8559696 


30.3247 


0.659233 


1.549317 


0.0137593 


8.1375 


1.627495 


3 . 824905 


0.0020492 


14.7948 


0.547957 


1.287797 


0.1526769 


19.5299 


0.527835 


1.240507 


0 . 1600404 


36.1636 


0. 645778 


1. 517696 


0.0114023 


3.4718 


0.578626 


1.359876 


0.2288249 


4 . 5586 


0.569829 


1.339200 


0.2212122 


6.5391 


0.435941 


1.024540 


0.4275257 


1.2651 


0.632559 


1.486627 


0.2270415 


14.0630 


0 . 484930 


1. 139672 


0.2827731 


2.2102 


0.736717 


1.731418 


0.1594515 


3.1879 


0.455411 


1.070298 


0.3811256 


234.0246 


0.425499 







Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


9.8503 


1.407189 


3.913888 


0.0003535 


12.2826 


0.944816 


2 . 627865 


0.0014467 


15.1026 


2 . 517097 


7.000932 


0.0000003 


6.7176 


1.119607 


3 . 114022 


0.0052210 


3.5477 


0.394188 


1.096376 


0.3634308 


2.8159 


2.815870 


7.831926 


0.0053132 


8.8346 


0.327206 


0.910074 


0.5979349 


8.7848 


0.274526 


0.763554 


0.8237228 


20.7928 


0.452016 


1.257217 


0.1253496 


10.1673 


2.033454 


5.655750 


0.0000426 


11.6354 


0.430941 


1.198599 


0.2264270 


17.7331 


0.479274 


1.333030 


0.0940917 


27.8577 


0.497460 


1.383610 


0.0390184 


5.1754 


0.862573 


2.399120 


0.0268520 


5.4297 


0.678707 


1.887723 


0.0595526 


6.9073 


0.460485 


1.280770 


0.2089887 


1.7807 


0.890356 


2 . 476393 


0.0849826 


14.6705 


0.505881 


1.407033 


0.0788824 


2.0737 


0.691223 


1.922534 


0.1248178 


3.0709 


0 . 438695 


1.220166 


0.2894603 


197.7456 


0.359537 







Df 

7 

13 

6 

6 

9 

1 

27 

32 

46 

5 

27 

37 

56 

6 

8 

15 

2 

29 

3 

7 

550 

Df 

7 

13 

6 

6 

9 

1 

27 

32 

46 

5 

27 

37 

56 

6 

8 

15 

2 

29 

3 

7 

550 

Df 

7 

13 

6 

6 

9 

1 

27 

32 

46 

5 

27 

37 

56 

6 

8 

15 

2 

29 

3 

7 

550 
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APPENDIX N. REDUCED SIMPLE MODELS 



Model: 

PA - factor (Squadron) + factor (Tot .Years) + factor (Shift ) 



scale: 0.3056853 



Df 

<none> 

factor (Squadron) 19 
factor (Tot .Years) 6 
factor (Shift ) 1 



Sum of Sq RSS Cp 

265.0655 281.5725 
19.09854 284.1640 289.0550 
6.98011 272.0456 284.8844 
2.98713 268.0526 283.9482 



Terms : 

Sum of Squares 
Deg. of Freedom 



factor (Squadron) 
16.4369 
19 



factor (Tot . Years) factor (Shift) Residuals 
7.2861 2.9871 265.0655 

6 1 866 



Residual standard error: 0.5532452 
Estimated effects may be unbalanced 



QA - factor (Squadron) + factor (Tot . Years) + factor (Shift ) 



scale: 0.4002939 



<none> 
factor ( Squadron) 
factor (Tot. Years) 
factor (Shift) 



Df Sum of Sq RSS Cp 

348.6818 370.2977 
19 23.06513 371.7469 378.1516 

6 15.45358 364.1354 380.9477 

1 0.90043 349.5822 370.3975 



Terms : 

Sum of Squares 
Deg. of Freedom 



factor (Squadron) 
22.4130 
19 



factor (Tot .Years) factor (Shift) Residuals 
15.4952 0.9004 348.6818 

6 1 866 



Residual standard error: 0.6345352 
Estimated effects may be unbalanced 



Model: 

RS ~ factor (Squadron) + factor(Rank) + factor (Tot . Years ) + factor (Shift ) 



scale: 0.3476029 



Df 

<none> 

factor (Squadron) 19 
factor (Rank) 6 
factor (Tot .Years) 6 
factor (Shift) 1 



Sum of Sq 

16.56125 

5.50006 

7.70716 

1.04953 



RSS Cp 
301.9366 324.8784 
318.4979 328.2308 
307.4367 326.2072 
309.6438 328.4143 
302.9861 325.2327 



Terms : 

Sum of Squares 
Deg. of Freedom 



factor (Squadron) 
16.0094 
19 



factor (Rank) factor (Tot . Years ) 
18.0278 7.6388 

6 6 



factor (Shift) Residuals 
1.0495 301.9366 

1 860 



Residual standard error: 0.5925277 
Estimated effects may be unbalanced 



Model: 

RM - factor (Squadron) + factor (Rank) + factor (Tot . Years ) 



scale: 0.3237372 



Df Sum of 

<none> 



Sq RSS Cp 

280.8129 301.5321 
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factor (Squadron) 19 
factor (Rank) 6 
factor (Tot. Years) 6 



23.32159 304.1345 312.5517 
5.07801 285.8909 302.7253 
10.54164 291.3546 308.1889 



Terms : 

Sum of Squares 
Deg . of Freedom 



factor (Squadron) 
27.8628 
19 



factor (Rank) 
19.2269 
6 



factor (Tot . Years ) 
10.5416 
6 



Residuals 

280.8129 

861 



Residual standard error: 0.5710932 
Estimated effects may be unbalanced 



Model : 

CC - factor (Squadron) + factor (Tot . Years) + factor (Shift ) 



scale: 0.4031287 



Df 

<none> 

factor (Squadron) 19 
factor (Tot . Years) 6 
factor (Shift ) 1 



Sum of Sq RSS Cp 

350.8909 372.6598 
22.16263 373.0535 379.5036 
17.43463 368.3255 385.2569 
2.80445 353.6953 374.6580 



Terms : 

Sum of Squares 
Deg. of Freedom 



factor (Squadron) 
22.0953 
19 



factor (Tot .Years) 
17.1855 
6 



factor (Shift ) Residuals 
2.8045 350.8909 

1 866 



Residual standard error: 0.636542 
Estimated effects may be unbalanced 



Model: 

CR ~ factor (Community) + factor (Tot . Years) + factor (Shift ) 



scale: 0.4686355 



<none> 
factor (Community) 
factor (Tot . Years) 
factor (Shift ) 



Df Sum of Sq RSS Cp 

418.7697 432.8288 
7 10.24208 429.0118 436.5100 

6 21.33850 440.1082 448.5437 

1 1.89642 420.6661 433.7879 



Terms : 

Sum of Squares 
Deg. of Freedom 



factor (Community) 
12.4423 
7 



factor (Tot .Years ) 
21.2783 
6 



factor (Shift ) Residuals 
1.8964 418.7697 

1 878 



Residual standard error: 0.690622 
Estimated effects may be unbalanced 
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APPENDIX O. REDUCED SIMPLE MODELS: SCATTERPLOTS WITH 
SUPERIMPOSED REGRESSION LINE 

PA QA 




Fined . Jacior(Squadron) ♦ lactor(Tot. Years) ♦ lactor(Shift) 




RS 



RM 





Fined • factor(Squadron) ♦ factor<Rank) ♦ fador(Tot. Years) ♦ lactor(Shift) 



Fined lactor( Squadron) ♦ tacior(Rank) ♦ taeiorfTot Years) 



cc 



CR 




3.2 3 4 3.6 3 8 4 0 4 2 




Fitted : fact or( Squadron) ♦ »aclor(Tot Years) ♦ faclor(Shitt) 



Fined : factor(Community) ♦ laclor(Tot.Years) * factor(Shtn) 
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100 200 300 0 100 200 300 0 100 200 300 



APPENDIX P. REDUCED SIMPLE MODELS: HISTOGRAMS OF RESIDUALS 



PA 




resid(pa.step) 

RS 




resid(rs.step) 



CC 




- 2-10 1 2 



resid(cc.step) 



QA 



A 



RM 



•1 o 1 

resid(qa.step) 



V\ 

ygr ’ T* 1 •’W 




CR 



resid(rm.step) 




- 2-1012 

resid(cr.step) 
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APPENDIX Q. REDUCED SIMPLE MODELS: RESIDUALS VERSUS FITTED 
VALUES 



PA 



QA 





Fitted factor{Squadron) + factor(Tot. Years) + taetor(Shrft) 



Fitted tactor(Squadron) * tactor{Tot. Years) * lacior(Shitt) 



RS 



RM 




<509 



3.0 3.2 3 4 3.6 3 8 4 0 4.2 

Fined lactor(Squadron) + tacior(Rank) ♦ Jactor(Tot.Years) ♦ lactor(Shift) 



o 



o o o o 




o o 

<£78 <569 



<589 



2 8 3.0 3.2 3 4 3.6 3.8 4.0 

Fitted factor(Squadron) ♦ factor(Rank) + tactor(Tot. Years) 



cc 



CR 





Fitted tactor(Squadron) + ( act or(Tot. Years) + factor(Shitt) 



Fitted faciorfCommumty) * faclor(Tot. Years) ♦ Jactorf Shift) 
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APPENDIX R. REDUCED SIMPLE MODELS: QQ-PLOTS 



PA 



QA 





RS 



RM 




CC 



CR 
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APPENDIX S. REDUCED TWO FACTOR INTERACTION MODELS 



Model : 

PA - factor (Community) + factor (Squadron) + factor (Tot .Years) + factor (Shift) + 
factor (Community) : factor (Shift) 

scale: 0.2751949 



<none> 
factor (Squadron) 
factor (Tot .Years) 
factor (Community) : factor (Shift) 



Df Sum of Sq RSS Cp 

259.7296 277.8925 
12 8.235337 267.9649 279.5231 

6 7.737269 267.4669 282.3274 

5 5.319236 265.0488 280.4597 



Model : 

QA - factor (Community) + factor (Squadron) + factor (Tot . Years) + factor (Shift) + 
factor (Community) : factor (Shift) 

scale: 0.3641075 



<none> 
factor (Squadron) 
factor (Tot . Years) 
factor (Community) : factor (Shift) 



Df Sum of Sq RSS Cp 

339.8904 363.9215 
12 12.36975 352.2601 367.5526 

6 16.34776 356.2381 375.8999 

5 8.42356 348.3139 368.7039 



Model : 

RS ~ factor (Squadron) + factor (Rank) + factor (Tot . Years) * factor (Shop) + factor (Shif t) 
factor (Squadron) : factor (Shift) + factor (Tot . Years) : factor (Shif t ) 



scale: 0.3104461 



factor (Squadron) 
factor (Tot .Years) 



<none> 
factor (Rank) 
factor (Shop) 
: factor (Shift) 
: factor (Shift) 



Df Sum of Sq 

280 

6 5.38955 286 

9 6.45093 287 

12 11.55724 292 

5 4.31919 285 



RSS Cp 
7776 317.4103 
1672 319.0745 
2286 318.2732 
3349 321.5168 
0968 318.6250 



Model : 

RM - factor (Squadron) + factor(Rank) + factor (Tot .Years) + factor(Shop) + factor (Shif t) 
factor (Squadron) : factor (Shif t) + factor (Tot . Years) : factor (Shif t) 

scale: 0.2949249 



<none> 
factor (Rank) 
factor (Shop) 
factor (Squadron) : factor (Shif t) 
factor (Tot .Years) : factor (Shif t) 



Df Sum of Sq 

6 3.99087 

9 5.55127 

12 17.43743 

5 3.57653 



RSS Cp 
255.4226 290.2237 
259.4134 290.6755 
260.9738 290.4663 
272.8600 300.5829 
258.9991 290.8510 



Model : 

CC - factor (Community) + factor (Squadron) + factor (Rank) + factor (Tot . Years ) 
factor (Shif t) + factor (Community) : factor (Shif t) + factor (Squadron) : factor (Shif t) + 

factor (Rank) : factor (Shif t) + factor (Tot . Years) : factor (Shif t) 

scale: 0.3595374 



Df Sum of Sq 



<none> 

factor (Community) : factor (Shif t) 0 

factor (Squadron) : factor (Shif t) 7 
factor (Rank) : factor (Shif t) 5 
factor (Tot . Years) : factor (Shif t) 5 



0.000000 

5.543296 

4.227122 

5.275932 



RSS Cp 
324.3285 364.5967 
324.3285 364.5967 
329.8718 365.1065 
328.5556 365.2285 
329.6045 366.2773 
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Model : 

CR - factor (Community ) + factor (Rank) + factor (Tot . Years ) + factor ( Shift ) + 
factor (Community) ; factor (Shift) + factor (Rank) : factor (Shift ) + factor (Tot .Years ) 
factor (Shift) 

scale: 0.4254993 



<none> 



Df Sum of Sq RSS Cp 

393 .4122 424.8992 



factor (Community) : factor (Shift) 



6 9.058145 402.4703 428.8513 
5 7.047982 400.4602 427.6921 
5 7.483000 400.8952 428.1272 



factor (Rank) : factor (Shi ft) 
factor (Tot .Years) : factor (Shift ) 
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APPENDIX T. REDUCED TWO FACTOR INTERACTION MODELS: 

SCATTERPLOTS WITH SUPERIMPOSED REGRESSION 
LINE 



PA 



QA 





Fined : factor(Commumty) + factor(Squadron) + fact or(Tot. Years) ♦ factor(Shift) * factor(Communily):1actor(J Fitted : factor(Community) + lactor(Squadron) 1actor(Tot. Years) + faclor(Shrtt) «■ factor(Coinmunity):lactor(« 



RS 



RM 





tadron) ♦ factor(Rank) fact or(Tot. Years) ♦ factor(Shop) + factor(Shrft) + factor(Squadron):factor(Shitt) + factonadron) + factor(Rank) ♦ lactor(Tot. Years) ♦ factor(Shop) + factor(Shift) ♦ factor(Squadron) factor(Shift) • faciot 



CC 



CR 





lrty) ♦ factor(Squadron) + factor(Rank) ♦ factor(Tot. Years) ♦ lactor(Shift) + factor(Communrty):factor(Shift) + faorty) factor(Rank) + factor(Tot. Years) + factor(Shift) ♦ factor(Community):factor(Shift) ♦ tactor(Rank).factor(Sf“ 
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100 200 300 0 100 200 300 



APPENDIX U. REDUCED TWO FACTOR INTERACTION MODELS: 
HISTOGRAMS OF RESIDUALS 
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APPENDIX V. REDUCED TWO FACTOR INTERACTION MODELS: 
RESIDUALS VERSUS FITTED VALUES 

PA QA 





Fitted tact or (Community) ♦ tactor(Squadron) ♦ tactor(Tot.Years) ♦ tactor(Shift) ♦ tador(Communrty):faetor(S Fitted tactor(Communrty) ♦ tactor(Squadron) «. t*ctor(Tot Years) ♦ tactOf(Shitt) ■* lador(Community):fador(S 



RS 



RM 





ladron) ♦ factor{Rank) ♦ tad orfTot. Years) ♦ tactor(Shop) ♦ factor(Shitt) + tador(Squadron);fador(Shift) ♦ factooadron) ♦ (actor(Rank) ♦ lactor(Tot. Years) ♦ factor(Shop) ♦ tactor(Shitt) ♦ tador(Squadron);fador(Shift) ♦ factor 



CC 



CR 





nty) «■ lactor(Squadron) + tador(Rank) + 1actor(Tot. Years) ♦ tador(Shitr) + 1ador(Community)-.1actof(Shitt) ♦faoity) ♦ tador(Rank) ♦ t3dor(Tot Years) ♦ fador(Shrft) ♦ fador(Community):1ador(Shitt) ♦ tador(RanK):1ador(St- 
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APPENDIX W. REDUCED TWO FACTOR INTERACTION MODELS: QQ- 
PLOTS 



PA 



QA 





RS 



RM 





CC 



CR 
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APPENDIX X. COMPARISONS ON MODELS USING ANOVA 



The values listed in the following tables are the probabilities that the models are 
equivalent. Values printed in bold type indicate similar models. 



Component 


Simple vs 
Two Factor 
Interaction 


Simple vs. 
Reduced 
Simple 


Simple vs. 
Reduced Two 
Factor 
Interaction 


PA 


0.00316 


0.37994 


0.99999 


QA 


0.00657 


0.17934 


0.99990 


RS 


0.00174 


0.04694 


0.00029 


RM 


0.00730 


0.09862 


1.86442e-7 


CC 


0.00156 


0.20517 


0.00001 


CR 


0.00580 


0.03360 


0.10600 



Component 


Two Factor 
Interaction 
vs. Reduced 
Simple 


Two Factor 
Interaction 
vs. Reducd 
Two Factor 
Interaction 


PA 


0.00324 


0.00788 


QA 


0.00488 


0.01596 


RS 


0.00087 


0.01474 


RM 


0.00470 


0.14662 


CC 


0.00121 


0.02202 


CR 


0.00200 


0.02111 



Component 


Reduced 
Simple vs. 
Reduced Two 
Factor 
Interaction 


PA 


0.00752 


QA 


0.00120 


RS 


0.00011 


RM 


3.50999e-7 


CC 


0.00008 


CR 


0.00016 
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APPENDIX Y. SUBJECT MATTER EXPERT RECOMMMENDED CHANGES 
TO MCAS DEMOGRAPHIC FACTORS 

Factors currently included in MCAS (no change): 

1 . Community 

2. Squadron 

3. Rank 

4. Work Center 

5. Shift 

Items to modify in MCAS: 

Total years of aviation maintenance experience (single item) 
modified to: 

6. Years worked in MOS 

7. Years worked outside MOS 

Factors to add to MCAS: 

education/training level 

8. attended an A school 

9. highest level of education attained 

10. number of maintenance activities assigned to during career 
morale/motivation indicators 

1 1 . command advanced 

12. number of personal awards 

13. level of job satisfaction 

14. time to attain qualifications 

15. past performance on personal evaluations 

16. assigned to B tour 

other factors 

17. Age 

18. number of months in current squadron 

19. number of deployments 

20. supervisory designations earned 

21. level of confidence in Maintenance Control Officer 
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